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Due to the criticality of continuous mission operations, some control centers must plan for alternate locations 
in the event an emergency shuts down the primary control center. Johnson Space Center (JSC) in Houston, 
Texas is the Mission Control Center (MCC) for the International Space Station (ISS). Due to Houston’s 
proximity to the Gulf of Mexico, JSC is prone to threats from hurricanes which could cause flooding, wind 
damage, and electrical outages to the buildings supporting the MCC. Marshall Space Flight Center (MSFC) 
has the capability to be the Backup Control Center for the ISS if the situation is needed. While the MSFC 
Huntsville Operations Support Center (HOSC) does house the BCC, the prime customer and operator of the 
ISS is still the JSC flight operations team. To satisfy the customer and maintain continuous mission 
operations, the BCC has critical infrastructure that hosts ISS ground systems and flight operations 
equipment that mirrors the prime mission control facility. However, a complete duplicate of Mission Control 
Center in another remote location is very expensive to recreate. The HOSC has infrastructure and services 
that MCC utilized for its backup control center to reduce the costs of a somewhat redundant service. While 
labor talents are equivalent, experiences are not Certain operations are maintained in a redundant mode, 
while others are simply maintained as single string with adequate sparing levels of equipment. Personnel at 
the BCC facility must be trained and certified to an adequate level on primary MCC systems. Negotiations 
with the customer were done to match requirements with existing capabilities, and to prioritize resources for 
appropriate level of service. Because some of these systems are shared, an activation of the backup control 
center will cause a suspension of scheduled HOSC activities that may share resources needed by the BCC. 
For example, the MCC is monitoring a hurricane in the Gulf of Mexico. As the threat to MCC increases, 
HOSC must begin a phased activation of the BCC, while working resource conflicts with normal HOSC 
activities. In a long duration outage to the MCC, this could cause serious impacts to the BCC host facility’s 
primary mission support activities. This management of a BCC is worked based on customer expectations 
and negotiations done before emergencies occur. 


I. Introduction 

P RIOR to 2007, when bad weather threatened the Houston, Texas area and Mission Control Center (MCC), the 
International Space Station (ISS) was controlled by the Backup Control Center (BCC) located in Moscow, 
Russia in the same building as Mission Control Center-Moscow. Besides the obvious logistical and cost 
disadvantages of maintaining a BCC in another country, JSC flight controllers also had limited contact with the ISS 
if there was no electrical power at the Mission Control Center in Houston. In that situation, the BCC was restricted 
to ground contact with the ISS while the Station is orbiting over Russia’s ground stations. 

With the ISS Program determined to lower its operating costs, program management decided to build a BCC in 
Huntsville, Alabama at the Marshall Space Flight Center (MSFC). There are three primary reasons why MSFC was 
chosen to host the new ISS BCC: secure remote access capability, direct access to Tracking and Data Relay Satellite 
System (TDRSS) ground station at White Sands, New Mexico, and geographic proximity to Houston. First, the 
Huntsville Operations Support Center (HOSC) hosts the ISS Payload Operations Integration Center (POIC). The 
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POIC, in consideration of lower operating costs, developed a secure remote access architecture for payload 
developers to acquire science telemetry and send commands to the POIC for uplink to the ISS. This prevents 
payload developers and science teams from having to spend long durations of time at the POIC in Huntsville. 
MSFC has a suite of personal computer based software applications that provide ground support systems capabilities 
for scientists named the Telescience Resource Kit (TReK) (http://trek.msfc.nasa.gov/about.htm). This software can 
be used to receive the payload data and to perform data processing on the local user’s systems or forward to other 
customer ground processors. MSFC also designed and implemented an Internet Voice Distribution System (IVoDS) 
for multi-conference voice loops availability on a personal computer. This allows users remote to the control center 
to talk and listen to mission voice loops. This system uses a remote user’s broadband network connection for voice 
connectivity to the POIC. It is also based on secure Internet Protocol (IP) protocols. During emergency 
preparations, a small flight control team has stationed in Round Rock, Texas, a suburb of Austin, in case hurricane 
winds started approaching Houston. This team has used IVoDS to monitor operations while in their temporary hotel 
location. Second, the POIC has a direct communications path to White Sands, New Mexico, just as JSC does. The 
POIC receives Ku-band telemetry, processes the science data, stores it, and forwards it to the appropriate science 
teams or control center. Both NASA centers have S-band ground connectivity via NASA Communications Network 
(NASCOM). So, a forward and return path between MSFC and White Sands exists just as it does with JSC. 
Finally, MSFC is approximately 975 kilometers from JSC. In case of a MCC shut down, or orderly evacuation of 
the Houston area, the flight control team has more options to travel to Huntsville. The team can either fly or drive to 
the BCC location. 

Initial Capability and Requirements Growth 

The original design for the BCC-HOSC was given in a proposal to NASA in 2004. After using Russian Ground 
Services for commanding during Hurricane Rita in September 2005, it was determined that the BCC should be 
relocated to the HOSC. The BCC-HOSC project scope was to augment the HOSC capability to support MCC in 
case of a catastrophic event. JSC commanding servers and other MCC support equipment would be placed in the 
POIC. The use of existing NASA wide area networks provided greater flexibility in distributing data and sourcing 
uplink data across NASA by utilizing this established infrastructure. As the host facility for the BCC, the POIC’s 
contributions to the project team included: 

• Assisted JSC with logistics and funding to relocate communications and encryption equipment to White 
Sands 

• Procured and installed JSC command servers in POIC per JSC design specifications 

• Configured room at the POIC for JSC controllers 

• Updated MSFC command database as required to support core commanding. This included mapping JSC 
commands to a specific user-ID for JSC personnel. These commands were also mapped to a specific 
POIC user-ID as a contingency user if required 

• Provided operations, maintenance and sustaining support as required. JSC would provide systems 
administration of JSC duplicate hardware. 

• Provided a virtual private network (VPN) or other secure interfaces to JSC for remote system 
administration of servers. 

• Provided engineering and documentation support to the team to implement project. 

JSC established support priorities in case of a MCC outage. These priorities are listed in order of importance: 
voice services, S-band commanding, and orbital communications adapter. The POIC had a priority of “keep science 
alive” in the case of the prime control center outage. Thus, POIC was more than willing to sacrifice facilities and 
scheduling priorities for a BCC. The POIC wanted to ensure that JSC flight directors could retain some level of 
commanding for control of the Ku-band antenna on ISS to ensure science data still flows down to the payload 
developers and scientists. While this design was not to completely match every aspect of the MCC, it quickly grew 
to provide a fully functional command and control capability for up to eleven flight control positions. Figure 1 
shows this initial architecture. 
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Figure 1. Backup Control Center - HOSC Generic Architecture 


The original concept of the BCC was only to support minimum crew, vehicle and system safety. Interaction 
between the crew and the ground was to be minimized. Complex operations, such as an extravehicular activity or 
docking maneuvers, were to be avoided while MCC was inactive and BCC was active. However, this new BCC- 
HOSC increased flight control support and approximated ground facilities as close to MCC as possible. During the 
shutdown of JSC due to Hurricane Ike in 2008, there was a Russian Progress vehicle docking at ISS. This was 
supported by the JSC flight team at the BCC-HOSC. Thus, the BCC had now proven to go beyond the original 
concept of the design. Additional requirements were added to match the capabilities of the primary control center in 
Houston, though at a smaller scale. New capabilities are being added in 2010 for additional support for International 
Partners. 


Hurricane Gustav and Ike Activations 

On August 28, 2008, the HOSC was notified of a potential BCC activation 
by JSC. Hurricane Gustav was a potential threat to the Houston area. This 
hurricane had already caused serious damage in Haiti, the Dominican Republic, 
Jamaica, and Cuba. HOSC personnel immediately started organizing staffing 
plans for the potential support. The first steps that must be taken involve remote 
communications support for the JSC flight control team. As they deploy to their 
temporary location in Round Rock, Texas, the HOSC team must ensure their 
successful, secure access to HOSC and BCC resources. Accounts are verified 
and personnel are ready for any communications trouble involving Internet 
communications between Texas and Alabama. Another process that must be 
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Figure 2. Hurricane Gustav 
Storm Track 







started is coordination with the MSFC Protective Services Office. Since MSFC resides within United States Army’s 
Redstone Arsenal, preparations are made to ensure the JSC team can quickly enter the Arsenal’s boundary with 
proper identification, and badges. Key cards are assigned for access to the HOSC and BCC resources, and car 
passes assigned for quick access through the Arsenal gates. This was all established by the next day, August 29, 
2008. On August 30, JSC decided to send the Flight Director to Huntsville and deploy the BCC remote team to 
Round Rock. That afternoon, HOSC established services to the Round Rock team. Later that night, JSC personnel 
arrived in Huntsville. Sunday, August 31, the BCC team at the HOSC was ready for activation of the facility. 
However, Gustav’s track moved north and bypassed the Houston area. Figure 2 shows the storm track of Gustav. 
JSC decided not to activate the BCC. JSC was not closed due to Gustav landfall in Louisiana and BCC teams in 
Round Rock and Huntsville were released to return home. The HOSC personnel worked continually through the 
Labor Day 2008 holiday week-end to ensure readiness until the decision was made to not activate the facility as the 
prime ISS Control Center. 

Right behind Gustav in the Atlantic Ocean was Hurricane Ike. One week after JSC and MSFC had returned to 

normal operations, BCC-HOSC systems were initiated and all HOSC 
configurations were frozen in anticipation of another Gulf of Mexico event. On 
September 10, 2008, once again, JSC deployed teams to Round Rock and 
Huntsville. However, this storm was decisively heading to Houston. Voice 
services were swung from JSC to MSFC. All voice loops originated from the 
HOSC. On September 11, MCC handed over control to the flight team at 
Round Rock. JSC was closing the center as an evacuation order had gone out to 

the greater Houston area. Figure 3 shows that Ike made landfall near Houston, 
Texas. However, MCC systems were remaining powered so as to allow 
International Partner connectivity for as long as possible. The Round Rock 
team was accessing systems at the HOSC as well as MCC through the HOSC’s secure remote access system. On 
September 12, the BCC-HOSC Flight Team was the prime control for voice and telemetry. The Flight Team at 
Round Rock was prime for commanding. However, they were reporting intermittent Internet connectivity due to 
high traffic demands in the Austin, Texas area. Additional MSFC desktop computer contractors were needed for 
support as the JSC flight team at MSFC was having e-mail issues accessing their mail servers. One of NASA’s 
locations for centralized e-mail services was at JSC. The Russian Progress supply ship was re-scheduled for 
docking at ISS on September 18 due to the transition to the BCC. 

MCC went to generator power on September 13 as power outages were widespread in the Houston area. Round 
Rock and Huntsville flight and support teams had been working twelve hour shifts. Round Rock team transferred 
commanding responsibilities to the Huntsville team on September 1 8. This released the Round Rock team to travel 
to Huntsville to add a third shift to BCC activities. BCC-HOSC was the prime ISS control center. The BCC-HOSC 
supported the Russian Progress 30P docking at ISS. MCC in Moscow required the docking video. As HOSC had 
no video path to Moscow, the HOSC could source the video to the European Space Agency (ESA). In the spirit of 
international cooperation, ESA then routed this video to MCC-Moscow. Normal ISS activities were supported until 
September 19, when HOSC-BCC transitioned prime control center responsibilities back to MCC. The HOSC 
supported BCC control center activities for close to five days. 



Figure 3. Hurricane Ike Storm 
Track 
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II. Services Provided 



Figure 4. Payload Control Area 2 

Facility Hosting 

The main service the backup control center hosting facility is providing is space. Space that is needed for 
equipment, space that is needed for control room functions, and space needed for other functions such as conference 
rooms, support rooms and other operations support. Floor space is usually allocated in a facility. Who can dedicate 
space to a backup control center that may never be used? There must be space available that can be shared or pre- 
empted for primary control center activities during the activation of the backup control center. Figure 4 is a picture 
of the Payload Control Area 2 (PCA2). It is currently used as a backup to the Payload Control Area 1 (PCA1), 
which is the ISS Payloads Operation Control Center (POCC). PCA2 is usually in use every day. When major 
system upgrades occur in PCA1, the POCC flight team moves to PCA2 to resume payload operations. Other uses of 
PCA2 include simulations, console and systems training, and operational readiness tests. This room also houses the 
BCC-HOSC consoles. Both HOSC and MCC computers at each console position share the keyboard, video display, 
and computer mouse using a hardware device that allows switching of the peripheral devices among the attached 
computers. This configuration saves on equipment costs and valuable console space. 

Other rooms are also made available to the BCC as needed during activation. For example, conference rooms 
are available for team meetings and discussions done outside of the control room. A MCC workstation is made 
available for video conferences with the astronauts. This console is in a smaller work area, and is used for private 
conferences with the crew. The equipment is usually in a non-intrusive area of the office during normal payloads 
operation. As with PCA2, these other rooms are pre-empted for MCC use during BCC activation. Space is also 
needed in the environmentally controlled computer rooms for servers and other computer equipment that service the 
MCC systems in the POIC. Racks must be designated or installed for the primary control center equipment as well 
as a work area for system upgrades testing or hardware fixes for MCC engineers and technicians during site visits. 

Another aspect of facility hosting is access to the information technology (IT) infrastructure of the host site. 
How the primary control center’s systems can benefit from the host’s IT infrastructure is a discriminator in selecting 
a location for an alternate control center. As mentioned earlier, POIC had developed a secure remote access for 
scientists and payload developers to interact with the HOSC for commanding and receiving telemetry from the 
POCC. Using a virtual private network (VPN), MCC personnel could interconnect with their systems at the HOSC 
for operations and maintenance. Routine operations such as upgrades, bug fixes, security patches, and software 
installations could be done remotely without travel to the BCC site. BCC-HOSC created its own network within the 
HOSC. As per design, this was to emulate the prime control center as much as possible. The BCC-HOSC network 

5 

American Institute of Aeronautics and Astronautics 




connected to the HOSC network at the same interface, or prime external router, that normal communications 
between JSC MCC and MSFC POIC occur. This philosophy easily replicates the security trust relationship 
between the two entities as the communications path between the two networks is duplicated as much as possible. 
Normal Internet connectivity is also provided for JSC tools that require external connectivity such as the JSC Mobile 
Computing System or MCC Automated System services. These mission support services are automated workflow 
tools that require connectivity to servers that are located at secondary locations outside of the Houston area. 

As mentioned earlier, one of the main advantages of hosting the BCC at MSFC was direct access to TDRSS 
ground stations. NASA mission network demarcation points are located in the HOSC. This gives MSFC and BCC 
direct access to S-band return link for telemetry data and Space to Ground voice communications. Ku-band forward 
and return link exist for MSFC payload telemetry data and Orbital Communications Adapter (OCA) for on-board 
services, data file transfer and video conferencing capability. BCC uses this link for Zone of Exclusion telemetry 
and four channels of ISS video. BCC uses the host facility connectivity for S-band uplink for commanding. Since 
JSC controls the uplink for commanding to the Station, this connectivity is critical for overall normalcy of 
operations during a JSC shut-down. 

Training and Certification 

The BCC host facility also provides personnel to assist the prime control center in maintaining and operating its 
equipment. It is impractical to travel to the BCC location every time equipment needs repair or replacement, so host 
BCC facility personnel are trained and certified to operate the prime control center equipment. Both JSC MCC and 
MSFC HOSC use similar operating systems on their servers. There are some tools that are shared between both 
control centers. However, each control center environment was independently designed and developed. HOSC 
personnel must be trained to a level of minimum certification of MCC responsibilities. MCC provided initial 
training for system controllers, system administrators, database administrators, and hardware maintenance 
technicians. The HOSC Security Officer and HOSC communications personnel were also trained to the added 
responsibilities as defined by a memorandum of agreement between the two centers. JSC also trained HOSC 
personnel to the appropriate level of certification as defined in other project documentation such as the BCC-HOSC 
User Handbook. 

HOSC personnel traveled to JSC MCC. They were immersed in the prime control center support environment 
and trained in normal maintenance activities. Training was provided for support roles in each of the operations 
support positions for normal troubleshooting, configurations, and daily operations and maintenance of the systems. 
Additional training was also provided to HOSC personnel while the MCC support staff was in Huntsville to do the 
initial installation and configuration of the new BCC-HOSC systems. HOSC then provided subsequent training to 
other local personnel on the BCC systems. This JSC provided training was integrated into the training and 
certification plan for the HOSC’s Operations Support Team. While HOSC personnel were not fully certified on 
MCC systems, they were minimally certified for first level troubleshooting, configuration management, card level 
replacement of equipment, and other duties as defined in operations agreements. HOSC personnel also directly 
support the BCC Facility Systems Manager (FSM) position which is staffed by JSC personnel during activation. 
Monthly proficiency testing is required using the BCC systems to ensure proficient knowledge of FSM procedures 
for successful direct support of this position. 

This certification is done annually and new training is given for any changes in procedures or equipment that has 
occurred during the previous year. HOSC personnel must be ready for BCC activation at any time, though hurricane 
season is when readiness and training proficiencies are at its peak. Hurricane season in the Atlantic Ocean is 
defined from June 1 to November 30 each year 1 . Because of personnel turnover, refresher training is always needed. 
This training introduces the BCC to new personnel at each location while also introducing the employees to then- 
contacts at the other site. Having this relationship established during training may allow for smoother operations 
during stressful BCC activation times. 

Operations and Maintenance 

Even if the BCC is not activated, there are daily activities associated with its systems that require attention and 
reporting to the primary control center. BCC systems in the HOSC are always powered, configured, and 
synchronized with the MCC. This keeps them in a ready state for activation in an emergency situation. As part of 
its sustaining responsibilities, HOSC provides the first line of maintenance action of BCC equipment. This usually 
consists of replacing failed equipment with a spare or replacing a failed part with a spare. Some pieces of equipment 
are quite expensive to have spare units local. Since the BCC is already a contingency facility, not all hardware has a 
spare unit stored in the facility. HOSC will coordinate with the vendor repair facility to send replacement units for 
these kinds of repairs. All BCC equipment failures are reported to the MCC Help Desk at JSC for tracking and 
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trending statistics. As part of its daily operations, HOSC personnel conduct daily checks of BCC systems health and 
status. A daily report of BCC systems status is provided to MCC. 

As part of the training program mentioned previously, two systems administrators provide system sustaining 
operations. This includes system backup and restoral from failures. As part of the agreement with JSC, these 
administrators are granted privileged access to the BCC systems. HOSC also does inventory and property 
management of BCC systems. As this equipment is still owned by JSC MCC, property management reports and 
inventory scanning results are provided to the MCC property custodians. 

HOSC also provides a Systems Security Officer (SSO) to coordinate with MCC SSO in the event of a security 
incident. The HOSC SSO performs an investigation using system audit information in BCC-HOSC systems. The 
HOSC SSO conducts this investigation using MSFC established security procedures. Results of any investigation 
are also shared with the MCC SSO. 

If HOSC needs to use the BCC-HOSC systems, this activity must be scheduled through the MCC Scheduling 
service. HOSC activities that use BCC systems include training activities, operations proficiency activities, and 
hardware replacement or repairs. JSC MCC schedules quarterly and annual testing of BCC-HOSC systems. HOSC 
personnel will notify MCC of any possible schedule conflicts. BCC systems are checked quarterly with an annual 
verification of system. This annual certification of BCC systems involves a mock activation. A small JSC flight 
control team will deploy to Round Rock and Huntsville to exercise the systems to verify connectivity and operations 
as documented. This test will also include a swing of the prime control center to BCC-HOSC for test command and 
telemetry responses with the International Space Station. Scheduling the annual test is difficult. There must be no 
conflicts with either control center or ISS activities. The JSC team then must coordinate the travel logistics for the 
two teams. The swing of the communications link is quick. Upon successful communications between BCC and 
ISS, the link is quickly swung back to JSC control. Other BCC systems’ functionality can be verified without 
connection to the ISS. 

Activation 

The POIC and MCC have different missions with respect to ISS operations. While MCC operates the core 
functions to sustain the Station, it is staffed to handle full shifts 24 hours a day, 7 days a week. The POIC directs 
payload science missions on the Station. Many payload experiments are self-sustaining and require little, if any, 
astronaut involvement. The POIC has a minimal staff to support off-shift hours, thus a minimum contingent of 
functions are staffed at a 24x7 level. The HOSC facility support is mainly staffed at an 8 hours a day, 5 days a week 
level. If a hurricane, or any event, threatens the closure of MCC, BCC-HOSC must be activated. The POIC facility 
must support the BCC-HOSC at an equal level of support as MCC. The requirement for BCC activation is to last no 
longer than 30 days. It is assumed that any emergency can be corrected and operations restored at MCC within this 
timeframe. Beyond the normal operations support team, which has been staffed at equivalent MCC levels, other 
functions in the HOSC must be staffed according to BCC activation requirements. System administrators, network 
managers, network and communications engineering, hardware maintenance and other direct facilities support must 
increase hours or additional people to meet this temporary elevation of support. 

HOSC personnel are trained to activate the BCC-HOSC in case of a sudden disconnect from JSC MCC. While 
the activation and swing of the prime control center to Huntsville is a coordinated process, it can be done, in an 
emergency, from Huntsville alone. A detailed procedure must be followed to reconfigure the BCC systems in 
Huntsville so as to stop being in backup mode primarily synchronizing data with the primary systems in Houston. 
The BCC systems are reconfigured to act as the primary control system servers. The BCC-HOSC then acts as 
Houston for connections with the ISS and MCC-Moscow. NASA Communications is notified. This is a simple 
example of what the system administrators do to make the BCC-HOSC systems function as the prime control center 
servers. 

Communication engineers and technicians coordinate with NASCOM at Goddard Space Flight Center to swing a 
predefined set of voice loops originating from JSC to originate from MSFC. This can be done at a set time not to 
interfere with the flight team in Houston. The service works normally, though the configuration of the loops has 
now changed. HOSC Communications will configure the needed loops on the voice instruments in the PCA2 room. 
These loops are also sourced to the IVoDS system so the remote teams can also utilize BCC voice services. 

NASCOM reconfigures the serial conversion device at MSFC to accept S-band data originally destined to JSC. 
The BCC-HOSC systems have the current MCC configurations to interpret this data just as if at JSC. Ku-band data 
from ISS is sent simultaneously to both JSC and MSFC. Under normal operations, MSFC only processes the 
science data. In BCC activation, all Ku-band data is processed. This includes ISS video data which is usually 
processed at JSC. The Ku-band ground network interface at MSFC must be configured to accept and process all 
data from Station. 
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At this point, all technical components are configured and operating as BCC acts in MCC mode. Technical 
requirements are satisfied. The facility’s primary mission has now changed to supporting core ISS operations. Any 
activity scheduled for rooms needed by the BCC flight team must be cleared for their use. 

III. Lessons Learned 

Additional Requirements 

Even after all the stress, time, and work done during Hurricane Ike, the ISS operated as if it were managed by 
normal ground operations. JSC and MSFC teams worked long hours in a new mode of operations to ensure 
continued mission success. Because this real time experience was a success, additional requirements and 
capabilities have been added or planned. As noted during the Hurricane Ike event, MCC in JSC remained powered 
to ensure International Partner capabilities were in service for as long as possible. The Japanese Space Agency 
(JAXA), the European Space Agency (ESA), and Russian Federal Space Agency (ROSCOSMOS) have all been 
connected to the BCC-HOSC to ensure their telemetry and commanding services remain during an emergency event 
in Houston. 

As the flight team now has supported ISS on-orbit from the BCC-HOSC, new requirements have been added to 
facilitate smoother operations during activation. Two more console positions will be added to PCA2. During 
Hurricane Ike activation, some team members had to share workstations. All BCC, HOSC, and MCC systems will 
be undergoing a change as more stringent security requirements have been levied by the Federal Government. Two 
factor authentication will be implemented on all mission systems. This will be done using RSA SecurlD tokens 
(http://www.rsa. com/node.aspx?id= 1156). The conversion of all workstation and application logins will be 
completed in 2011. 

Maintaining Operational Readiness State 

Typically the prime control center marches to their own schedules when trying to synchronize their BCC system 
configuration changes with their primary operations systems. This typically involves making sure database versions 
and application software versions are synchronized, then tested the readiness conditions. The BCC host must pay 
constant attention to system change schedules published by the prime control center. This is necessary to avoid 
scheduling conflicts. The BCC host must maintain a relationship with the prime’s planning function as this is 
imperative to ensure scheduling conflicts do not become unnecessary reworks of activities at both control centers. 

Maintaining Training and Certification 

The BCC control center’s training material standards, operations procedures standards, and certification 
programs are never standardized with prime control center’s processes. The prime organization has enough 
problems maintaining their own programs without having to maintain two sets of standards, not to mention the 
impacts to the people who have to train, certify, or actually operate systems. Integration of the owning 
organization’s training, certification, and operations materials into the BCC host’s organizations processes is 
essential to effective host support to the owning organization. The hosting organization will have no confusion of 
training or processes as the training programs are integrated. Of course, this does create an additional burden on the 
host organization of regenerating, training, certification, and procedure materials to conform to the host 
organizations standards but in the long run reduces the impact to the host organization when the back up control 
center has to be activated. 

Shared Resource Impacts 

Always in an effort to reduce systems cost, training, and operations, the prime control center organization 
looks to take advantage of using existing systems and services in the host BCC facility as much as possible. This 
works well if the owner can make use of the existing host services and systems with no modifications. Problems 
arise when an attempt is made to integrate systems and services between the two organizations. Impacts become 
significant when either organization requires upgrades to the systems/services that have been integrated. For 
example, at the HOSC, simple things such as video monitor refresh activities had the potential to impact BCC 
systems because of an incompatibility with MCC workstation video cards. 

Unexpected Additional Support Issues 

“Hey would you mind doing us a favor because we forgot to... and we need to have it done today or we 

can’t ” become the norm when the owning organization achieves a comfort zone with host organization’s ability 

to maintain systems and services. Typically this takes place after budgets and associated support funding has already 
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been negotiated. Best lesson learned is to clearly define roles and responsibilities and adhere to them in all 
circumstances and to make sure that role/responsibilities are always reviewed for currency, especially when system 
enhancements are made. The HOSC learned a valuable lesson on maintenance agreements. The HOSC agreed to 
maintain a specific set of hardware until a software upgrade was made and tested. The systems would then be 
transferred to the owning organization. Therefore a contract maintenance agreement was not renewed. As luck 
would have it, the last unit failed during the software upgrade verification test. The result was an unplanned $8500 
repair bill for the HOSC. 

Upgrade Impacts 

The prime control center organization typically uses their own requirement change process for enhancing 
their BCC systems and services. These processes typically are not well suited for outsider involvement. Impacts 
include the host organization’s ability to identify cost and schedule impacts back to the prime organization’s formal 
requirements change process. Results typically involve after the fact negotiations that never work to the host’s 
advantage. It is imperative that the host organization, when defining role and responsibilities include the need to be 
integrated into the prime organization’s formal requirement change process. The process also needs to clearly define 
the responsibility of the host organization when effecting the required change. 

Logistics Impacts 

Host organizations are typically required to store and maintain BCC spare resources. Hosting this service 
typically involves finding a place to store the hardware, regularly scheduled audits, and associated shipping and 
receiving logistics. These are typical logistics items that are accounted for in support negotiations. Identifying 
responsibility for acquiring vendor maintenance support for systems and verification of spare item condition is 
usually not a typical logistical task. The prime organization typically has maintenance contracts with support 
vendors and those contracts can be proprietary in nature. The host organization has a difficulty acquiring 
maintenance support if they do not have that information available. This is typically not a big impact when just 
sustaining the BCC system from a day to day basis as the prime organization can take on the responsibility for 
calling in trouble reports to the vendor. The problem arises when the BCC is activated and those contract 
agreements are not available. Arrangements must be made between the prime organization and the host so that 
maintenance support can be acquired quickly. Also, spare hardware, sitting for long periods of time, may become 
non-operational for numerous reasons: battery depletion, firmware updates, etc. A plan needs to be devised to ensure 
the readiness state of the spare units. One way to satisfy this requirement is to put the spare in to the operations 
environment on a recurring basis to limit the risk. The host organization should ensure that this kind of activity be 
clearly delineated in negotiated roles and responsibilities as mentioned above. 

IV. Conclusion 

The host of the BCC should have a vested interest in the success of the mission and the resumption of normal 
operations as quickly as possible. This strengthens the teamwork between the prime and host organizations. The 
POIC would be inoperable if MCC shut down in previous times. One of the normal procedures that MCC does 
before switching to an alternate location was to shut down the Ku-band antenna on ISS. This would suspend science 
capabilities during the emergency event on the ground. Flight operations were kept to a minimum during the 
contingency operation. Having a fully capable BCC at the HOSC ensures normal operations for the POIC and the 
continuous flow of science data even during MCC shutdowns. During a BCC activation, the HOSC works with the 
flight team to ensure their comfort and success during a stressful time. Operating a BCC requires deferring 
resources to the primary control center to keep the HOSC mission alive. 
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